Scheduling tasks

ABSTRACT

A method of activating scheduling instructions within a parallel processing unit is described. The method includes checking if an ALU targeted by a decoded instruction is full by checking a value of an ALU work fullness counter stored in the instruction controller and associated with the targeted ALU. If the targeted ALU is not full, the decoded instruction is sent to the targeted ALU for execution and the ALU work fullness counter associated with the targeted ALU is updated. If, however, the targeted ALU is full, a scheduler is triggered to de-activate the scheduled task by changing the scheduled task from the active state to a non-active state. When an ALU changes from being full to not being full, the scheduler is triggered to re-activate an oldest scheduled task waiting for the ALU by removing the oldest scheduled task from the non-active state.

BACKGROUND

A graphics processing unit (GPU) comprises a highly parallel structurewhich is designed to efficiently process large amounts of data inparallel. GPUs are typically used for computer graphics (e.g. to render3D images on a screen), however they may also be used for otheroperations which benefit from the parallelism they provide.

The embodiments described below are provided by way of example only andare not limiting of implementations which solve any or all of thedisadvantages of known GPUs or other parallel processing units.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

A method of activating scheduling instructions within a parallelprocessing unit is described. The method comprises checking if an ALUtargeted by a decoded instruction is full by checking a value of an ALUwork fullness counter stored in the instruction controller andassociated with the targeted ALU. If the targeted ALU is not full, thedecoded instruction is sent to the targeted ALU for execution and theALU work fullness counter associated with the targeted ALU is updated.If, however, the targeted ALU is full, a scheduler is triggered tode-activate the scheduled task by changing the scheduled task from theactive state to a non-active state. When an ALU changes from being fullto not being full, the scheduler is triggered to re-activate an oldestscheduled task waiting for the ALU by removing the oldest scheduled taskfrom the non-active state.

A first aspect provides a method of activating scheduling instructionswithin a parallel processing unit comprising: decoding, in aninstruction controller, an instruction in a scheduled task in an activestate; checking if an ALU targeted by the decoded instruction is full bychecking a value of an ALU work fullness counter stored in theinstruction controller and associated with the targeted ALU; in responseto determining that the targeted ALU is not full, sending the decodedinstruction to the targeted ALU for execution and updating the ALU workfullness counter associated with the targeted ALU; in response todetermining that the targeted ALU is full, triggering a scheduler tode-activate the scheduled task by changing the scheduled task from theactive state to a non-active state; monitoring the ALU work fullnesscounters stored in the instruction controller; and in response todetecting, as a consequence of the monitoring, that an ALU has changedfrom being full to not being full, triggering the scheduler tore-activate an oldest scheduled task waiting for the ALU by removing theoldest scheduled task from the non-active state.

A second aspect provides an instruction controller comprising: an inputfor receiving an instruction in a scheduled task in an active state froma scheduled task scheduler; an instruction decoder arranged to decodethe received instruction; a memory element arranged to store a pluralityof ALU work fullness counters; and hardware logic arranged to check ifan ALU targeted by the decoded instruction is full by checking a valueof the ALU work fullness counter associated with the targeted ALU, andin response to determining that the targeted ALU is not full, to sendthe decoded instruction to the targeted ALU for execution and update theALU work fullness counter associated with the targeted ALU; hardwarelogic arranged, in response to determining that the targeted ALU isfull, to trigger the scheduler to de-activate the scheduled task bychanging the scheduled task from the active state to a non-active state;and hardware logic arranged, to monitor the ALU work fullness countersand in response to detecting, as a consequence of the monitoring, thatan ALU has changed from being full to not being full, to trigger thescheduler to re-activate an oldest scheduled task waiting for the ALU byremoving the oldest scheduled task from the non-active state.

A third aspect provides an integrated circuit manufacturing systemcomprising: a computer readable storage medium having stored thereon acomputer readable description of an integrated circuit that describes aninstruction controller; a layout processing system configured to processthe integrated circuit description so as to generate a circuit layoutdescription of an integrated circuit embodying the instructioncontroller; and an integrated circuit generation system configured tomanufacture the instruction controller according to the circuit layoutdescription, wherein the instruction controller comprises: an input forreceiving an instruction in a scheduled task in an active state from ascheduler; an instruction decoder arranged to decode the receivedinstruction; a memory element arranged to store a plurality of ALU workfullness counters; and hardware logic arranged to check if an ALUtargeted by the decoded instruction is full by checking a value of theALU work fullness counter associated with the targeted ALU, and inresponse to determining that the targeted ALU is not full, to send thedecoded instruction to the targeted ALU for execution and update the ALUwork fullness counter associated with the targeted ALU; hardware logicarranged, in response to determining that the targeted ALU is full, totrigger the scheduler to de-activate the scheduled task by changing thescheduled task from the active state to a non-active state; and hardwarelogic arranged, to monitor the ALU work fullness counters and inresponse to detecting, as a consequence of the monitoring, that an ALUhas changed from being full to not being full, to trigger the schedulerto re-activate an oldest scheduled task waiting for the ALU by removingthe oldest scheduled task from the non-active state.

Further aspects provide: a parallel processing system configured toperform the method as described herein; computer readable codeconfigured to perform the steps of the method described herein when thecode is run on a computer; a method of manufacturing, using anintegrated circuit manufacturing system, an instruction controller asdescribed herein; computer readable code configured to cause the methodas described herein to be performed when the code is run; an integratedcircuit definition dataset that, when processed in an integrated circuitmanufacturing system, configures the integrated circuit manufacturingsystem to manufacture an instruction controller as described herein; acomputer readable storage medium having stored thereon a computerreadable description of an integrated circuit that, when processed in anintegrated circuit manufacturing system, causes the integrated circuitmanufacturing system to manufacture an instruction controller asdescribed herein; an integrated circuit manufacturing system configuredto an instruction controller as described herein.

The instruction controller and/or scheduled task scheduler describedherein may be embodied in hardware on an integrated circuit. There maybe provided a method of manufacturing, at an integrated circuitmanufacturing system, an instruction controller and/or scheduled taskscheduler as described herein. There may be provided an integratedcircuit definition dataset that, when processed in an integrated circuitmanufacturing system, configures the system to manufacture aninstruction controller and/or scheduled task scheduler as describedherein. There may be provided a non-transitory computer readable storagemedium having stored thereon a computer readable description of anintegrated circuit that, when processed, causes a layout processingsystem to generate a circuit layout description used in an integratedcircuit manufacturing system to manufacture an instruction controllerand/or scheduled task scheduler as described herein.

There may be provided an integrated circuit manufacturing systemcomprising: a non-transitory computer readable storage medium havingstored thereon a computer readable integrated circuit description thatdescribes the instruction controller and/or scheduled task scheduler asdescribed herein; a layout processing system configured to process theintegrated circuit description so as to generate a circuit layoutdescription of an integrated circuit embodying the instructioncontroller and/or scheduled task scheduler as described herein; and anintegrated circuit generation system configured to manufacture theinstruction controller and/or scheduled task scheduler as describedherein according to the circuit layout description.

There may be provided computer program code for performing any of themethods described herein. There may be provided non-transitory computerreadable storage medium having stored thereon computer readableinstructions that, when executed at a computer system, cause thecomputer system to perform any of the methods described herein.

The above features may be combined as appropriate, as would be apparentto a skilled person, and may be combined with any of the aspects of theexamples described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

Examples will now be described in detail with reference to theaccompanying drawings in which:

FIG. 1 is a schematic diagram of an example processor;

FIG. 2 is a flow diagram of a method of synchronizing a group ofscheduled task in a processor;

FIG. 3 is a schematic diagram of another example processor;

FIG. 4 is a flow diagram of an example method of scheduling scheduledtasks within a processor;

FIG. 5 is a flow diagram of a further part of the example method ofscheduling scheduled tasks as shown in FIG. 4;

FIG. 6 is a schematic diagram of another example processor;

FIG. 7 is a flow diagram of another example method of schedulingscheduled tasks within a processor;

FIG. 8 is a flow diagram of a further part of the example method ofscheduling scheduled tasks as shown in FIG. 7;

FIG. 9 is a schematic diagram of yet another example processor;

FIG. 10 is a flow diagram of yet another example method of schedulingscheduled tasks within a processor;

FIG. 11 shows a computer system in which an instruction controllerand/or scheduled task scheduler as described herein are implemented; and

FIG. 12 shows an integrated circuit manufacturing system for generatingan integrated circuit embodying an instruction controller and/orscheduled task scheduler as described herein.

The accompanying drawings illustrate various examples. The skilledperson will appreciate that the illustrated element boundaries (e.g.,boxes, groups of boxes, or other shapes) in the drawings represent oneexample of the boundaries. It may be that in some examples, one elementmay be designed as multiple elements or that multiple elements may bedesigned as one element. Common reference numerals are used throughoutthe figures, where appropriate, to indicate similar features.

DETAILED DESCRIPTION

The following description is presented by way of example to enable aperson skilled in the art to make and use the invention. The presentinvention is not limited to the embodiments described herein and variousmodifications to the disclosed embodiments will be apparent to thoseskilled in the art.

Embodiments will now be described by way of example only.

Various methods and apparatus for scheduling within a GPU or otherparallel processing unit (such as for high performance computingapplications) are described herein. In particular, the methods describedherein relate to scheduling of tasks once all their dependencies havebeen met and they have all the resources required to run.

The term ‘task’ is used herein to refer to a group of data-items and thework that is to be performed upon those data-items. For example, a taskmay comprise or be associated with a program or reference to a program(e.g. the same sequence of ALU instructions or reference thereto) inaddition to a set of data that is to be processed according to theprogram, where this set of data may comprise one or more data elements(or data-items, e.g. a plurality of pixels or vertices).

The term ‘program instance’ is used herein to refer to individualinstances that take a path through the code. A program instancetherefore refers to a single data-item and a reference (e.g. pointer) toa program which will be executed on the data-item. A task thereforecould be considered to comprise a plurality of program instances (e.g.up to 32 program instances), though in practice only a single instanceof the common program (or reference) is required per task. Groups oftasks that share a common purpose, share local memory and may executethe same program (although they may execute different parts of thatprogram) or compatible programs on different pieces of data may belinked by a group ID. A group of tasks with the same group ID may bereferred to as a ‘work-group’ (and hence the group ID may be referred toas the ‘Work-group ID’). There is therefore a hierarchy of terminology,with tasks comprising a plurality of program instances and groups (orwork-groups) comprising a plurality of tasks.

The methods described herein relate to two layers of scheduling tasks,the first layer of task scheduling being performed once all theirdependencies have been met and they have all the resources required torun, in order to form a set of “scheduled tasks”. Tasks that arescheduled may spawn more than one “scheduled task” (e.g. for examplewhere multi-sample anti-aliasing, MSAA, is used a single task mayschedule 2, 4, 8 or more scheduled tasks depending upon the particularMSAA rate). The methods herein also relate to scheduling, in the secondlayer of scheduling, one or more of the scheduled tasks from the set ofcurrently scheduled tasks to form a set of active tasks from thescheduled tasks, where the active tasks are to be executed by theparallel processor and may be a proper subset of the scheduled tasks.Therefore the methods may be described as method for schedulingscheduled tasks and the methods may be implemented by a scheduled taskscheduler (which is distinct from a task scheduler which initiallyschedules tasks). The scheduled task scheduler may be part of a largerscheduler which comprises both the scheduled task scheduler and a taskscheduler which is arranged to schedule scheduled tasks for executionbefore they have all the resources required to run. Tasks are scheduledinitially by the task scheduler (and are scheduled only once by the taskscheduler, unless it is a multi-phase task) and once a task is scheduled(and becomes a scheduled task), the corresponding scheduled tasks may bescheduled many times by the scheduled task scheduler. In particular,there may be many scheduled tasks and only a proper subset of thesescheduled tasks may be active (i.e. running and executing in aprocessing block) at any time. Consequently scheduled tasks may bescheduled (i.e. become active) and de-scheduled (e.g. by being placedinto one or more ‘waiting states’ where they are not active) many timesby the scheduled task scheduler before a task is completed. For the sakeof clarity and brevity, reference to the scheduling of scheduled tasks(by the scheduled task scheduler) will be referred to as “activating” or“re-activating” scheduled tasks (as the case may be) and thede-scheduling of scheduled tasks (by the scheduled task scheduler) willbe referred to as “de-activating” scheduled tasks. Accordingly anyreference to activation, deactivation or reactivation may be consideredto be a reference to the scheduling of scheduled tasks for execution.

When a task is received by the task scheduler which schedules tasks, thereceived task is scheduled and is added to a queue (which may bereferred to as a scheduled task queue) and is now ready to be selected(e.g. activated) by the scheduled task scheduler and executed (and hencethe scheduled task becomes active). When a scheduled task is active,instructions from the scheduled task are sent to an instruction decoderto be decoded and then the decoded instructions are passed to theappropriate ALU for execution.

Each scheduled task in the scheduled task scheduler has associated statedata which identifies the current state of the scheduled task, where ascheduled task may be active (i.e. executing on a processing blockwithin the GPU or other parallel processing unit) or not active (i.e.not executing on a processing block within the GPU or other parallelprocessing unit). Whilst there may only be one possible active state, invarious examples, there may be a plurality of not active states. Invarious examples there may be at least two distinct not active states: a‘ready’ state and one or more ‘waiting’ states. A scheduled task in theready state is available to be selected by the scheduled task schedulerfor execution and once selected (i.e. activated) the scheduled taskwould move from the ready state into the active state. A scheduled taskin a waiting state, in contrast, is not available to be selected by thescheduled task scheduler and a waiting state has associated criteriawhich specify when the scheduled task can be placed back into the readystate. In examples where there are different waiting states, these mayhave different associated criteria and various examples are described inthe different methods described below. A waiting state may also bereferred to as a de-activated state, as typically a scheduled task isplaced into a waiting state when it is de-activated (i.e. when it isremoved from the active state for some reason) and hence stops beingexecuted by the processing block.

The state data for each scheduled task may be stored with the scheduledtask in the scheduled task queue (e.g. where there is a single queuewhich stores scheduled tasks in various different states, as identifiedby the state data for each scheduled task). In other examples there maybe multiple queues of scheduled tasks, with each queue corresponding toa particular state and comprising only the scheduled tasks that are inthat state (e.g. an active queue comprising only those scheduled tasksin the active state, and one or more not active queues each comprisingscheduled tasks in a different one of the not active states).

The number of active scheduled tasks in the scheduled task scheduler isa proper subset of the total number of scheduled tasks in the scheduledtask scheduler. In various examples the number of active scheduled tasksis determined by the latency of an instruction decoder within theprocessing block multiplied by the number of instruction decoders, e.g.such that if the latency of the instruction decoder is 7 clock cyclesand there are two instruction decoders, there will be 14 activescheduled tasks. Once the maximum number of active scheduled tasks isreached, another scheduled task cannot become active until one of thecurrently active scheduled tasks is de-activated (e.g. by being placedinto a waiting state or into the ready state). Once the number of activescheduled tasks falls below the maximum permitted number, the scheduledtask scheduler selects a scheduled task to become active and in variousexamples, the scheduled task scheduler selects the oldest scheduled taskin the scheduled task scheduler which is in the ready state to becomeactive. Selection of the oldest scheduled task to become active is oneexample of a scheduled task selection scheme (i.e. activation scheme)and in other examples, different schemes may be used.

Described herein are various methods and apparatus for scheduling (e.g.activating, deactivating and/or reactivating) scheduled tasks within aGPU or other parallel processing unit. Although the methods describedherein are described as being implemented in hardware, at least one ofthe methods described herein enables software to control, or at leastinfluence, the activation process and the methods may alternatively beimplemented, at least partially, in software (e.g. by replacing ahardware state machine with a programmable sequencer which executesmicrocode that implements the state machine functionality). In themethods described herein, the scheduled task scheduler activatesscheduled tasks which are in a ready state based on pre-definedcriteria, such as age-based criteria (as described above). Thescheduling is then modified by the scheduled task scheduler or by theinstruction controller which selectively triggers the de-activation ofscheduled tasks, i.e. by causing the scheduled task scheduler to place ascheduled task either back into the ready state or into a waiting state.Depending upon the method described, there may be a number of possiblewaiting states into which a scheduled task is placed and the currentstate of any scheduled task may be recorded using state data storedwithin the queue of scheduled tasks or by moving the scheduled task tothe appropriate queue (e.g. where different queues in the scheduled taskscheduler correspond to the different possible waiting states).Additionally, depending upon the method described, a scheduled task maybe de-activated based on the workload of the target ALU (or ALUpipeline) where the workload may be defined in terms of a number ofinstructions or a number of scheduled tasks that can send instructionsto the target ALU pipeline.

Also described herein are methods and apparatus for synchronizing agroup of scheduled tasks into a known state within a GPU or otherparallel processing unit. In various applications (e.g. OpenCL) thesynchronization process may be referred to as a work-group barrier andso the methods and apparatus described herein may be used to implementwork-group barriers. However, the methods and apparatus are not limitedto OpenCL and are also applicable to other compute APIs (e.g. HSA and DXcompute).

The methods for synchronizing a group of scheduled tasks into a knownstate may be implemented by a scheduled task scheduler and an ALU withinthe GPU or other parallel processing unit. The scheduled task scheduleruses a waiting state referred to herein as a sleep state and a newinstruction which is executed by the ALU to synchronize scheduled taskswith the same group ID. The methods described avoid the need to use alock (where a lock only allows one scheduled task to progress at a timethrough sections of code protected by a lock), reduce software overhead(as a single instruction is used) and are faster (as the methods areimplemented predominantly in hardware).

Although the different methods are described separately in the followingdescription, it will be appreciated that the methods may be implementedindependently or any two or more of the methods described herein may beimplemented together.

Methods and apparatus for synchronizing a group of scheduled taskswithin a GPU or other parallel processing unit can be described withreference to FIGS. 1-2.

FIG. 1 is a schematic diagram showing a processor 100 which may be a GPUor other parallel processing unit. It will be appreciated that FIG. 1only shows some elements of the processor and there may be many otherelements (e.g. caches, interfaces, etc.) within the processor that arenot shown in FIG. 1. The processor 100 comprises a scheduler 102, aninstruction decoder 103 and a processing block 104.

The processing block 104 comprises hardware logic for executing theinstructions within scheduled tasks that are scheduled for execution bythe scheduler 102 and which have been decoded by the instruction decoder103. The processing block 104 therefore comprises many arithmetic logicunits (ALUs) and the ALUs may be grouped in any way. The processingblock 104 may comprise different types of ALUs, e.g. with each type ofALU being optimized for a particular type of computation. In exampleswhere the processor 100 is a GPU, the processing block 104 may comprisea plurality of shader cores, with each shader core comprising one ormore ALUs. In various examples, the processing block 104 may be asingle-instruction multi-data (SIMD) processor (which may in variousexamples it may be referred to as a Unified Shading Cluster (USC)) or asingle-instruction single-data (SISD) processor.

The scheduler 102 comprises a first (or task) scheduler 106 and a second(or scheduled task) scheduler 108. As described above, tasks aregenerally scheduled only once by the first scheduler 106 (unless a taskis a multi-phase task); however, once a task is scheduled (and becomes ascheduled task or multiple scheduled tasks, e.g. in the case of MSAA),the corresponding scheduled task(s) may be scheduled many times by thesecond (scheduled task) scheduler 108. In particular, there may be manyscheduled tasks which correspond to tasks and only a proper subset ofthese scheduled tasks may be active (i.e. running and executing in theprocessing block 104) at any time. Consequently scheduled tasks may beactivated (i.e. become active) and de-activated (e.g. by being placedinto one or more ‘waiting states’ where they are not active) by thesecond scheduler 108 many times before a scheduled task is completed.

As shown in FIG. 1, the processing block 104 comprises an ALU pipeline112, referred to as an ‘atomic ALU pipeline’ which is used tosynchronize groups of scheduled tasks as described in more detail below.The atomic ALU pipeline 112 may be dedicated to the purpose ofsynchronizing groups of scheduled tasks or may additionally performother atomic operations and in various examples there may be more thanone atomic ALU pipeline 112. Each group of scheduled tasks has anassigned area of local memory and this is used by the atomic ALUpipeline 112 to store data that it uses to perform the synchronizationof scheduled tasks within a group.

As shown in FIG. 1, the scheduler 102 receives tasks 110 and the firstscheduler 106 selectively schedules these tasks 110 for execution by theprocessing unit 104. Once a task is scheduled by the first scheduler 106all its dependencies will have been met and it has the requiredresources allocated to it. The scheduled task(s) corresponding to thetask are then selectively activated and de-activated by the secondscheduler 108.

FIG. 2 is a flow diagram of an example method of synchronizing a groupof scheduled tasks in a processor 100 (which may be a GPU or otherparallel processing unit) as shown in FIG. 1. The second scheduler 108activates scheduled tasks and sends instructions from activatedscheduled tasks to be decoded by the instruction decoder 103 and thenexecuted by ALUs within the processing block 104. As shown in FIG. 2,when the second scheduler 108 sends a particular type of instruction,referred to herein as a synchronization instruction, for decoding (block202), the second scheduler 108 receives, in response, an indication fromthe instruction decoder 103 to place the particular scheduled task intoa sleep state and so the scheduled task is placed into the sleep state(block 204). The decoded synchronization instruction is sent to theatomic ALU pipeline 112 (by the instruction decoder 103) and the decodedsynchronization instruction comprises the group identifier (ID) of thescheduled task to which the synchronization instruction relates orotherwise identifies the group to which the scheduled task belongs. Invarious examples, the instruction may also identify the particularscheduled task in which the synchronization instruction has beenreceived (e.g. by means of a scheduled task ID).

By putting a scheduled task into a sleep state (in block 204), thescheduled task is de-activated by the second scheduler 108. Whilst inthe sleep state (which is an example of a waiting state), a scheduledtask cannot be re-activated (e.g. based on age-based criteria or othercriteria). In various examples a second scheduler 108 may implementdifferent types of waiting state, each of which has pre-definedconditions that determine when the scheduled task can be removed fromthe waiting state. For the purposes of synchronizing a group ofscheduled task, scheduled tasks are placed into a waiting state referredto as a sleep state and cannot exit that state (and hence bere-activated) until a message is received from the atomic ALU pipelinewhich identifies the group ID of the scheduled task.

In response to receiving an instruction from the instruction decoder 103identifying a particular group of scheduled tasks (e.g. by means of thegroup ID) and optionally identifying a particular scheduled task withinthe group (e.g. by means of a scheduled task ID), the atomic ALUpipeline 112 performs an operation on data stored in a data store (i.e.an area of local memory) assigned to the particular group (block 206).

Having performed the operation (in block 206), the atomic ALU pipeline112 performs a check on the data stored in the data store assigned tothe particular group (block 208) and this check may comprise comparingthe data to a pre-defined value (e.g. comparing the value of the counterto a target value which may be one or N, where N is an integer) orvalues (e.g. checking whether all the bits in the store have been set,where each bit corresponds to one scheduled task in the group).

In various examples, the operation that is performed (in block 206) maycomprise incrementing or decrementing a counter. For example, if thereare N scheduled tasks within a group, a counter may initially be set to0 and then the operation (in block 206) may increment this counter eachtime an instruction is received which relates to the particular group.In such an example, the check which is performed may be to compare thecounter value to a target value of N. Alternatively, the counter mayinitially be set to N and the operation (in block 206) may decrementthis counter each time an instruction is received which relates to theparticular group. In such an example, the check which is performed maybe to (in block 208) compare the counter value to a target value ofzero.

In other examples, the operation that is performed (in block 206) maycomprise setting a bit corresponding to the scheduled task ID. Forexample, if there are N scheduled task within a group, the operation mayset a bit in the data store for the group each time an instruction isreceived which relates to the particular group. In such an example, thecheck which is performed (in block 208) may be to compare the storeddata to see if all the bits have been set. Alternatively, the operationmay store a scheduled task ID in the data store for the group each timean instruction is received which relates to the particular group andwhich includes a scheduled task ID (or otherwise identifies a particularscheduled task). In such an example, the check which is performed (inblock 208) may be to compare the stored data to see if all the requiredscheduled task IDs (or the right number of scheduled task IDs) have beenstored.

If the check (in block 208) is not passed (‘No’ in block 210) becausethe data stored does not match the target value(s), then no furtheraction is taken by the atomic ALU pipeline 112 at this stage.

If, however, the check (in block 208) is passed (‘Yes’ in block 210)because the data does match the target value(s), then the atomic ALUpipeline 112 sends a message to the second scheduler 108 whichidentifies the particular group to which it relates (block 212), i.e.the message identifies the group to which the check which passedrelates. The atomic ALU pipeline 112 may additionally reset the datastored in the data store assigned to the group of scheduled tasks (block213) e.g. by clearing the data stored or by resetting the counter tozero or N.

In response to receiving a message from the atomic ALU pipeline 112identifying a group, the second scheduler 108 removes all scheduledtasks for the identified group from the sleep state (block 214). Thismeans that these scheduled tasks can now be rescheduled immediately orat any point subsequently (e.g. using any suitable method and criteria).In various examples, when exiting a sleep state a scheduled task will beavailable to be activated according to another activation methodimplemented by the second scheduler 108.

Although in the examples described above the instructions and othermessages sent between the second scheduler 108 and the atomic ALUpipeline 112 identify a group (e.g. by means of a group ID), in otherexamples, there may be separate atomic ALU pipelines 112 for each groupand so the instructions and other messages sent between the secondscheduler 108 and the atomic ALU pipeline 112 inherently identify agroup of scheduled tasks (by means of either the source or destinationof an instruction or other message) and so do not need to include agroup ID or other identifier.

By using the method shown in FIG. 2 and described above, all thescheduled tasks in a group exit from the sleep state (and hence areavailable to be rescheduled) at the same time. This means that the datastored in the data store assigned to the group of scheduled tasks (asupdated by the atomic ALU pipeline 112) is no longer required and can beover-written (e.g. to perform a subsequent synchronization operation forthe same group of scheduled tasks) or re-allocated (e.g. to a differentgroup of scheduled tasks). Furthermore, as the atomic ALU pipeline 112performs the update on the data (in block 206) and the check on theupdated data (in block 208) in response to a single instruction (thesynchronization instruction), there is no need for a lock. This isbecause the operations are inherently serialized (i.e. the operationsare always executed sequentially) and there is no possibility thatanother instruction can over-write the data in the data store in betweenthe update operation (in block 206) and the check on the data (in block208).

Although the method shown in FIG. 2 can be used to synchronize all thescheduled tasks in a group (where a group comprises a collection ofscheduled tasks with the same group ID), in other examples, the methodmay alternatively (or in addition) be used to synchronize a propersubset of the scheduled tasks within a group. For example, a propersubset of the scheduled tasks may be synchronized by setting the valueof N (which may be the initial counter value or the target countervalue, as described above) to the number of scheduled tasks which needto be synchronized. Using this technique, any number of subsets may besynchronized with divergent synchronization points between the subsetsby providing each subset with its own data store.

By using the method shown in FIG. 2 and described above, thesynchronization of scheduled tasks is implemented predominantly inhardware (by the atomic ALU pipeline 112 and second scheduler 108) andso operates more quickly and reduces the complexity of the software code(e.g. compared to known methods which require many instructions toimplement the synchronization of work-items). Furthermore, use of asingle instruction rather than multiple instructions reduces thesoftware overhead.

Using the method shown in FIG. 2 and described above, a single atomicALU pipeline 112 may perform synchronization for multiple groups.

Methods and apparatus for scheduling (e.g. activating, deactivatingand/or reactivating) scheduled tasks within a GPU or other parallelprocessing unit which prevents ALU pipeline stalls can be described withreference to FIGS. 3-5.

FIG. 3 is a schematic diagram showing a processor 300 which may be a GPUor other parallel processing unit. It will be appreciated that FIG. 3only shows some elements of the processor and there may be many otherelements (e.g. caches, interfaces, etc.) within the processor that arenot shown in FIG. 3. The processor 300 comprises a scheduler 302, aninstruction decoder 303 (which is part of an instruction controller 320)and a processing block 304.

The scheduler 302 comprises a first scheduler 106 which schedules tasksand a second scheduler 308 which schedules (e.g. activates, deactivatesand/or reactivates) scheduled tasks. As described above, tasks aregenerally scheduled only once by the first scheduler 106; however, oncea task is scheduled, the corresponding scheduled tasks may be scheduled(e.g. activated or reactivated) many times by the second scheduler 108.In particular, there may be many scheduled tasks and only a propersubset of these scheduled tasks may be active (i.e. running andexecuting in the processing block 304) at any time. Consequentlyscheduled tasks may be activated (i.e. become active) and de-activated(e.g. by being placed into one or more ‘waiting states’ where they arenot active) many times before the processing of a scheduled task iscompleted.

The second scheduler 308 comprises one or more scheduled task queues 310which store scheduled tasks 312. In various examples the secondscheduler 308 comprises a single scheduled task queue 310 and this queue310 also stores state data 314 for each of the stored scheduled tasks312 (such that the state data 314 for a scheduled task 312 may bedescribed as being associated with a scheduled task 312) and the statedata 314 for a scheduled task 312 identifies the current state that thescheduled task 312 is in, e.g. whether it is in an active, ready orwaiting state and where there are multiple waiting states, whichparticular waiting state it is currently in. The state data 314 for ascheduled task 312 may also comprise flag data where the flag may be setor not set for a scheduled task and the operation of this flag isdescribed below. Alternatively the flag data may be stored within theinstruction controller 320. In other examples, the second scheduler 308comprises multiple queues, each queue corresponding to a different stateand storing scheduled tasks which are in the particular state (e.g. a‘ready queue’, an ‘active’ queue and one or more ‘waiting’ queues).These multiple queues may also comprise the flag data or the flag data(where provided) may be stored within the instruction controller 320.

The processing block 304 comprises hardware logic for executing theinstructions within tasks that are scheduled for execution by thescheduler 302. The processing block 304 therefore comprises a pluralityof arithmetic logic unit (ALU) pipelines 316 and the ALU pipelines maybe grouped in any way. The processing block 304 may comprise differenttypes of ALU pipelines 316, e.g. with each type of ALU pipeline beingoptimized for a particular type of computation. Each ALU pipeline 316comprises a queue 318 which stores a queue of work to be performed bythe ALU pipeline (i.e. one or more instructions to be executed by theALU pipeline 316). In examples where the processor 300 is a GPU, theprocessing block 304 may comprise a plurality of shader cores, with eachshader core comprising one or more ALU pipelines 316. In variousexamples, the processing block 304 may be a single-instructionmulti-data (SIMD) processor (which may in various examples it may bereferred to as a Unified Shading Cluster (USC)) or a single-instructionsingle-data (SISD) processor.

The instruction controller 320 comprises one or more instructiondecoders 303 and a plurality of ALU work fullness counters 322.

As shown in FIG. 3, the scheduler 302 receives tasks 110 and the firstscheduler 106 selectively schedules these tasks 110 for execution by theprocessing unit 304. The scheduled tasks 312 are stored in the scheduledtask queue 310 by the second scheduler 308 (or in a ready queue, wherethere are multiple queues) and then scheduled (e.g. activated,deactivated and/or reactivated) by the second scheduler 308. When ascheduled task 312 is activated by the second scheduler 308, its statechanges from a not active state (e.g. a ready state) to an active stateand it is sent to the instruction controller 320. When a scheduled taskchanges state, its state data is updated or the scheduled task may bemoved between queues to indicate the change in state.

The instruction controller 320 receives active scheduled tasks from thesecond scheduler 308 and decodes instructions associated with the activescheduled tasks using the instruction decoder 303. The instructioncontroller 320 then sends the decoded instructions to the targeted ALUpipeline 316 (i.e. to the ALU pipeline 316 which can execute theinstruction). The instruction controller 320 uses the plurality of ALUwork fullness counters 322 to track how much work each ALU pipeline 316has and the operation of these counters 322 and the instructioncontroller 320 are described in more detail below.

FIG. 4 is a flow diagram of an example method of scheduling scheduledtasks within a GPU or other parallel processing unit which reduces (oreliminates) ALU pipeline stalls in a processor 300 (which may be a GPUor other parallel processing unit) as shown in FIG. 3. The secondscheduler 308 activates scheduled tasks and sends instructions fromactivated tasks to be decoded by the instruction controller 320 (by theinstruction decoder 303) and then executed by ALU pipelines 316 withinthe processing block 304). As shown in FIG. 4, in response to receivingan instruction for decoding (block 402), the instruction controller 310decodes the instruction (block 404, in the instruction decoder 303) andthen checks whether the ALU pipeline that will execute the decodedinstruction (referred to as the target ALU pipeline) is full (block406), where this check uses the ALU work fullness counter 322 for thetarget pipeline.

If it is determined (by the instruction controller 320) that thescheduled task to which the instruction relates does not have a flag set(‘No’ in block 405) and that the target ALU pipeline is not full (‘No’in block 406), then the decoded instruction is sent to the target ALUpipeline 316 (block 408) and the ALU work fullness counter 322 for thetarget pipeline is updated (block 410), e.g. the ALU work fullnesscounter 322 for the target pipeline is updated (e.g. incremented). Theoperation where the scheduled task has the flag set (‘Yes’ in block 405)is described subsequently.

If, however, it is determined (by the instruction controller 320) thatthe scheduled task to which the instruction relates does not have a flagset (‘No’ in block 405) and that the target ALU pipeline is full (‘Yes’in block 406), then the instruction controller 320 triggers thede-activation of the scheduled task (block 412) and the decodedinstruction is discarded (i.e. it is not sent to the target ALUpipeline). The de-activation is performed by the second scheduler 308(in response to a message from the instruction controller 320) andcomprises switching the scheduled task from an active state into awaiting state (e.g. as recorded in the state data 314 associated withthe scheduled task 312 or by moving the scheduled task between differentqueues 310). The scheduled task is switched into a waiting state whichmay be referred to as a ‘pipeline full state’ for the particular targetALU pipeline or a ‘waiting for pipeline X’ state, where the Xcorresponds to the particular target ALU pipeline (e.g. one of ALUpipelines A-F in the example shown in FIG. 3). Consequently, whenimplementing this method, there may be multiple waiting states that canbe identified by the state data 314 within the queue 310 or multiplewaiting queues (one for each waiting state), where the multiple waitingstates comprise at least a ‘waiting for pipeline X’ state for aplurality (e.g. each) of the ALU pipelines in the processing block 304.In other examples, there may be a single ‘pipeline full’ (or ‘waitingfor pipeline’) state with the particular pipeline that any scheduledtask is waiting for being identified in sideband data. This sidebanddata is sent for a scheduled task from the instruction controller 320 tothe scheduler 308 when the instruction controller 320 de-activates ascheduled task (e.g. in block 412).

The second scheduler 308 monitors the ALU work fullness counters 322associated with any scheduled tasks 312 in the queue 310 that are in apipeline full' state (blocks 414-416). For example, if the queue 310comprises one scheduled task in a ‘pipeline full’ state which is waitingfor pipeline A and three scheduled tasks in a ‘pipeline full’ statewhich are waiting for pipeline F, then the second scheduler 308 monitorsthe ALU work fullness counters 322 for ALU pipelines A and F (in blocks414-416). In various examples, the instruction controller 320communicates the values of the ALU work fullness counters 322 to thesecond scheduler 308 (e.g. the instruction controller 320 may broadcastthe values) to enable the second scheduler 308 to perform themonitoring.

If the second scheduler 308 (whilst monitoring counters in blocks414-416) identifies that the ALU work fullness counter 322 for aparticular ALU pipeline (e.g. ALU pipeline X) is no longer full (‘Yes’in block 416), the second scheduler 308 updates the state of a scheduledtask that is waiting for the particular pipeline (block 420) e.g. ascheduled task which is waiting for ALU pipeline X. Where there is morethan one scheduled task which is waiting for the particular pipeline(e.g. ALU pipeline X), then the state of only one scheduled task isupdated (in block 420) and the oldest scheduled task in the ‘pipelinefull’ state that is waiting for the particular pipeline (e.g. the oldestscheduled task in the ‘pipeline full’ state which is waiting for ALUpipeline X) is first selected (block 418) and then the state of theselected scheduled task is updated (in block 420). As well as updatingthe state of a scheduled task for the particular ALU pipeline (in block420), a flag associated with the selected scheduled task is set (inblock 420) and the work fullness counter 322 for the particular ALUpipeline is updated (block 410), e.g. the counter 322 for the particularALU pipeline is incremented.

By incrementing the counter (in block 410) and setting the flag (inblock 420), the particular ALU pipeline resource is pre-allocated to theselected scheduled task and this gives priority to the next decodedinstruction from the selected scheduled task over decoded instructionsfrom other scheduled tasks which target the same ALU pipeline. Referringback to the start of FIG. 4, if an instruction is received (in block402) and decoded (in block 404) which relates to a scheduled task thathas the flag set (‘Yes’ in block 405), then the fullness check (in block406) is omitted. This is because the ALU work fullness counter 322 forthe target pipeline will already have been updated (in block 410 of aprevious execution of the method of FIG. 4) and the decoded instructionis sent to the target ALU pipeline (in block 408) irrespective of thecurrent value of the work fullness counter for the target ALU pipeline.In contrast, if an instruction is decoded (in block 404) which targetsthe same ALU pipeline but which relates to a scheduled task that doesnot have its flag set (‘No’ in block 405), the fullness check (in block406) is performed and the decoded instruction will only be sent to thetarget ALU pipeline if the work fullness counter indicates that thetarget ALU pipeline is not full (‘No’ in block 406). As shown in FIG. 4,a check for a flag is performed (block 409) prior to updating the ALUwork fullness counter (in block 410), to avoid the counter beingincremented twice for the same instruction (i.e. such that the counteris not updated if the flag is set, block 411).

For example, if ALU pipeline A is determined to no longer be full (inblock 416) and scheduled task 0 is currently in ‘pipeline full’ stateand waiting for pipeline A, scheduled task 0 is selected (in block 418),rescheduled and a flag is set for scheduled task 0 (in block 420). TheALU work fullness counter 322 for pipeline A is updated (in block 410)and this may mean that ALU pipeline A appears full again. If aninstruction which targets ALU pipeline A is received (in block 402) fora different scheduled task, scheduled task 1, which does not have itsflag set, before the next instruction for scheduled task 0 which targetsALU pipeline A, the incrementing of the ALU work fullness counter 322(in block 410) will prevent the instruction from scheduled task 1 beingsent to ALU pipeline A. Consequently the instruction from scheduledtask1 is prevented from causing ALU pipeline A to be full again and fromblocking progression of the instruction from scheduled task 0.

The change of state of the selected scheduled task (in block 420) mayswitch the scheduled task from a waiting state (e.g. a waiting forpipeline X state) to a ready state, such that the scheduled task canthen be selected for execution by the second scheduler 308 oralternatively the scheduled task may be switched directly from thewaiting state to an active state.

In various examples, the updating of the ALU work fullness counters 322(in block 410) comprises incrementing a counter and the fullness check(in block 406) may comprise determining if the ALU work fullness counterfor the target pipeline equals (or exceeds) a value that indicatesfullness. This value that indicates fullness may be the same for all ALUpipelines (and denoted F) or may be different for different ALUpipelines (and denoted F(X), where X refers to the particular ALUpipeline, such that in the example shown in FIG. 3 there are valuesF(A), F(B), F(C) . . . F(F)). The ALU work fullness counter for apipeline does not solely increase but may be decremented as shown inFIG. 5. When an instruction is removed from an ALU pipeline queue 318(block 502), the instruction controller 320 updates the ALU workfullness counter for the particular pipeline (block 504), e.g. bydecrementing the counter.

When determining whether an ALU pipeline is no longer full (in block416), the same values (e.g. F or F(X)) may be used or alternatively adifferent value may be used. For example, if the updating of the ALUwork fullness counters 322 as decoded instructions are sent to an ALUpipeline (in block 410) comprises incrementing a counter, a differentvalue F−Δ or F(X)−Δ may be used to provide hysteresis (i.e. such thatthe ALU pipeline does not show as being not full immediately that itswork fullness counter falls).

Alternatively, the updating of the ALU work fullness counters 322 asdecoded instructions are sent to an ALU pipeline (in block 410) maycomprise decrementing a counter. In such examples, the counter isincremented when an instruction is removed from the ALU pipeline queue(in block 502). In such examples, the fullness check (in block 406) maycomprise determining if the ALU work fullness counter for the targetpipeline equals (or is less than) a value that indicates fullness. Asdescribed above, this value that indicates fullness may be the same forall ALU pipelines (and denoted F) or may be different for different ALUpipelines (and denoted F(X)). When determining whether an ALU pipelineis no longer full (in block 416), the same values (e.g. F or F(X)) maybe used or alternatively a different value F+Δ or F(X)+Δ may be used toprovide hysteresis (i.e. such that the ALU pipeline does not show asbeing not full immediately that its work fullness counter increases).

Although the method of FIG. 4 is described above with reference to ALUpipelines, the method may also be applied for ALUs which are notpipelined (i.e. where instructions execute in a single clock cycle).

In various examples, the instruction controller 320 may store multiplecounters for each ALU pipeline 316, where the multiple counters for anALU pipeline 316 comprise an ALU work fullness counter (as describedabove) and an ALU deactivated scheduled tasks counter. Where ALUdeactivated scheduled tasks counters are used, they provide a count ofhow many scheduled tasks have been deactivated due to a particular ALUpipeline being full. As shown in FIG. 4, the ALU deactivated scheduledtasks counter for a particular pipeline is updated (block 413), e.g.incremented, each time a scheduled task is de-activated (in block 412)because the particular pipeline is considered full (‘Yes’ in block 406)and is updated in the opposite direction (block 422), e.g. decremented,each time a scheduled task is removed from the waiting state (in block420) because the particular pipeline is no longer considered full (‘Yes’in block 416). Where the ALU deactivated scheduled tasks counter isused, the test for fullness (in block 406) may be modified such that thevalue of the work fullness counter is combined with the deactivatedscheduled tasks counter before being compared to the value thatindicates fullness (F or F(X)). This assists in de-activating scheduledtasks earlier (i.e. at lower values of the ALU work fullness counters)and prioritizing those scheduled tasks which have previously beende-activated when an ALU pipeline has capacity (as indicated by its ALUwork fullness counter). Use of the second counter, i.e. the deactivatedscheduled tasks counter, may result in hardware which runs faster andoccupies a smaller area.

Using the methods and apparatus for scheduling scheduled tasks within aGPU or other parallel processing unit as described above with referenceto FIGS. 3-5, ALU pipeline stalls can be prevented. An ALU pipelinestall occurs when the second scheduler 308 sends scheduled tasks to beexecuted to the same ALU pipeline at a rate faster than the ALU pipelinecan execute. This results in a backlog of scheduled tasks to be executedby the ALU pipeline and can cause the scheduler 302 to stall (whichresults in a drop in performance of the GPU or other parallel processingunit). By using the ALU work fullness counters described above andupdating the counters when instructions are sent to an ALU pipeline andwhen instructions are removed from the queue in an ALU pipeline,scheduled tasks cannot be sent to an ALU pipeline at a rate faster thanthey can be accepted by the ALU pipeline queues 318.

Although the method is described above as using both flags (which may bereferred to as ‘pipeline full flags’) and a pipeline fullness counter,in various examples, the pipeline full flags maybe used independently ofthe pipeline fullness counter. However, by using the both the flags andthe pipeline fullness counters (as shown in FIG. 4 and described above),the power consumption of the apparatus is reduced. This is because ifflags are used in isolation (i.e. without the pipeline fullnesscounters), instructions from all scheduled tasks waiting for aparticular pipeline would be sent to the instruction controller 320 oncethe flag for the pipeline indicated the pipeline was no longer full. If,however, the pipeline does not have the capacity to accept all of theseinstructions, all the scheduled tasks that cannot be accommodated willthen be de-activated again. The unnecessary reactivation of scheduledtasks that are then immediately de-activated results in unnecessarypower consumption.

Methods and apparatus for optimizing the activation of scheduled taskswithin a GPU or other parallel processing unit by prioritizing a subset(i.e. a proper subset) of the scheduled tasks can be described withreference to FIGS. 6-8.

FIG. 6 is a schematic diagram showing a processor 600 which may be a GPUor other parallel processing unit. It will be appreciated that FIG. 6only shows some elements of the processor and there may be many otherelements (e.g. caches, interfaces, etc.) within the processor that arenot shown in FIG. 6. The processor 600 comprises a scheduler 602, aninstruction decoder 303 (which is part of an instruction controller 620)and a processing block 604.

The scheduler 602 comprises a first scheduler 106 which schedules tasksand a second scheduler 608 which schedules (e.g. activates, deactivatesand/or reactivates) scheduled tasks. As described above, tasks aregenerally scheduled only once by the first scheduler 106; however, oncea task is scheduled, the corresponding scheduled task(s) may beactivated many times by the second scheduler 608. In particular, only aproper subset of these scheduled tasks may be active (i.e. running andexecuting in the processing block 604) at any time. Consequentlyscheduled tasks may be activated or de-activated (e.g. by being placedinto one or more ‘waiting states’ where they are not active) andreactivated many times before a task is completed.

The second scheduler 608 comprises one or more queues 310 that storescheduled tasks 312 which correspond to tasks which have been scheduledby the first scheduler because they have the required resources (e.g.registers, memory, etc.) and their dependencies have been met. Invarious examples the second scheduler 608 comprises a single queue 310and this queue 310 also stores state data 314 for each of the storedscheduled tasks 312 (such that the state data 314 for a scheduled task312 may be described as being associated with a scheduled task 312) andthe state data 314 for a scheduled task 312 identifies the current statethat the scheduled task 312 is in, e.g. whether it is in an active,ready or waiting state and where there are multiple waiting states,which particular waiting state it is currently in. In other examples,the second scheduler 608 comprises multiple queues, each queuecorresponding to a different state and storing scheduled tasks that arein the particular state (e.g. a ‘ready queue’, an ‘active’ queue and oneor more ‘waiting’ queues).

The processing block 604 comprises hardware logic for executing theinstructions within scheduled tasks that are scheduled for execution bythe scheduler 602. The processing block 604 therefore comprises aplurality of ALU pipelines 316 and the ALU pipelines may be grouped inany way. The processing block 604 may comprise different types of ALUpipelines 316, e.g. with each type of ALU pipeline being optimized for aparticular type of computation. In examples where the processor 600 is aGPU, the processing block 604 may comprise a plurality of shader cores,with each shader core comprising one or more ALU pipelines 316. Invarious examples, the processing block 604 may be a single-instructionmulti-data (SIMD) processor (which may in various examples it may bereferred to as a Unified Shading Cluster (USC)) or a single-instructionsingle-data (SISD) processor.

Of the ALU pipelines 316 within the processing block 604, one may bedenoted the primary instruction pipeline (PIP). This may, for example,be the most used ALU pipeline (e.g. the one which overall reads andwrites the most data) and hence is the ALU pipeline that is ideally runmost efficiently. The instruction controller 620 comprises one or moreinstruction decoders 303 and a PIP Running List 622. The PIP runninglist 622 lists those scheduled tasks which are currently active on thePIP (i.e. those scheduled tasks having instructions currently executingon the PIP) and the size of the PIP running list 622 is fixed, i.e.there is a maximum number of scheduled tasks that can be included in thePIP running list 622 and hence a maximum number of scheduled tasks thatcan have instructions executing on the PIP concurrently. In variousexamples the same ALU pipeline may always be denoted the PIP and inother examples, different ALU pipelines may be denoted the PIP atdifferent times. Furthermore, although FIG. 6 shows a single PIP runninglist 622 and hence only one of the ALU pipelines 316 is denoted the PIP,in other examples, the instruction controller 620 may comprise multiplePIP running lists, each PIP running list corresponding to a differentPIP (e.g. PIP A, PIP B, . . . ) and the operation of each PIP runninglist is as described below.

As shown in FIG. 6, the scheduler 602 receives tasks 110 and the firstscheduler 106 selectively schedules these tasks 110 for execution by theprocessing unit 604 to form scheduled tasks. The scheduled tasks 312 arestored in a queue 310 by the second scheduler 608 and then scheduled(e.g. activated, deactivated and/or reactivated) by the second scheduler608. When a scheduled task 312 is activated by the second scheduler 608,its state changes from a not active state (e.g. a ready state) to anactive state and it is sent to the instruction controller 620.

The instruction controller 620 receives active scheduled tasks from thesecond scheduler 608 and decodes instructions associated with the activescheduled tasks using the instruction decoder 303. The instructioncontroller 620 then sends the decoded instructions associated with theactive scheduled tasks to the targeted ALU pipeline 316 (i.e. to the ALUpipeline 316 which can execute the instruction). The instructioncontroller 620 uses the PIP running list 622 to control whetherinstructions can be executed on the PIP and the operation of the PIPrunning list 622 and the instruction controller 620 are described inmore detail below.

FIG. 7 is a flow diagram of an example method of scheduling (e.g.activating, deactivating and/or reactivating) scheduled tasks within aGPU or other parallel processing unit which optimizes the scheduling(e.g. activation) of scheduled tasks by prioritizing a proper subset ofthe scheduled tasks (i.e. those scheduled tasks in the PIP running list622). As shown in FIG. 6, the second scheduler 608 activates scheduledtasks and sends instructions from activated scheduled tasks to bedecoded by the instruction controller 620 and then executed by ALUpipelines 316 within the processing block 604. As shown in FIG. 7, inresponse to receiving an instruction for decoding (block 402), theinstruction controller 620 decodes the instruction (block 404, in theinstruction decoder 303) and then checks whether the ALU pipeline thatwill execute the decoded instruction (referred to as the target ALUpipeline) is the PIP (block 706).

If it is determined (by the instruction controller 620) that the targetALU pipeline is not the PIP (‘No’ in block 706), then the decodedinstruction is sent to the target ALU pipeline 316 (block 408). If,however, it is determined (by the instruction controller 620) that thetarget ALU pipeline is the PIP (‘Yes’ in block 706), then theinstruction scheduler 620 checks the PIP running list 622 to determinewhether the scheduled task to which the instruction relates is alreadyincluded in the PIP running list 622 (block 710).

If it is determined (by the instruction controller 620) that thescheduled task to which the instruction relates is already in the PIPrunning list 622 (‘Yes’ in block 710), then the decoded instruction issent to the target ALU pipeline 316 (block 408), i.e. the decodedinstruction is sent to the PIP. If, however, it is determined (by theinstruction controller 620) that the scheduled task to which theinstruction relates is not already in the PIP running list 622 (‘No’ inblock 710), a check is performed to see whether the scheduled task towhich the instruction relates can be added to the PIP running list 622(block 712). As described above, there is a maximum number of scheduledtasks that can be included within the PIP running list 622 and if thelist is not full (‘No’ in block 712), i.e. the maximum number ofscheduled tasks in the PIP running list 622 has not been reached, thenthe scheduled task to which the decoded instruction relates is added tothe PIP running list 622 (block 714) and then the decoded instruction issent to the target ALU pipeline 316 (block 408), i.e. the decodedinstruction is sent to the PIP.

If, however, the PIP running list 622 is full (‘Yes’ in block 712), i.e.the maximum number of scheduled tasks in the PIP running list 622 hasbeen reached, then the scheduled task to which the decoded instructionrelates cannot be added to the PIP running list 622 and instead theinstruction controller 620 triggers the de-activation of the scheduledtask (block 716) and the decoded instruction is discarded. Thede-activation is performed by the second scheduler 608 in response tothe trigger from the instruction controller 620 and comprises switchingthe scheduled task from an active state back into a waiting state (e.g.as recorded in the state data 314 associated with the scheduled task 312or by moving the scheduled taskbetween queues). The scheduled task isswitched into a waiting state which may be referred to as a ‘PIP waitingstate’. This waiting state may be considered to be similar to the‘pipeline full’ or ‘waiting for pipeline X’ state described above withreference to FIGS. 3-5, where in this example X identifies the PIP.However, there is a difference in the mechanism which causes a scheduledtask to be placed into this state, because in the method shown in FIG.4, it is the number of instructions sent to the ALU pipeline that ismonitored and limited, whereas in this method shown in FIG. 7, it is thenumber of scheduled tasks that can send instructions to a particular ALUthat is monitored and limited.

The second scheduler 608 monitors the fullness of the PIP running list622 (blocks 718-720). In various examples, the instruction controller620 communicates the current number of scheduled task in the PIP runninglist 622 to the scheduler 602 (e.g. the instruction controller 620 maybroadcast this number) to enable the second scheduler 608 to perform themonitoring.

If the second scheduler 608 (whilst monitoring the number of entries inthe PIP running list in blocks 718-720) identifies that the PIP runninglist is no longer full (‘Yes’ in block 720, e.g. because the number ofentries has fallen below the maximum number), the second scheduler 608updates the state of a scheduled task that is waiting for the PIP (block724) e.g. a scheduled task which is in the PIP waiting state. Wherethere is more than one scheduled task which is in the PIP waiting state,then the state of only one scheduled task is updated (in block 724) andthe oldest scheduled task in the pipeline that is in the PIP waitingstate is first selected (block 722) and then the state of the selectedscheduled task is updated (in block 724).

The change of state of the selected scheduled task (in block 724) mayswitch the scheduled task from a waiting state (i.e. the PIP waitingstate) to a ready state, such that the scheduled task can then beselected for execution by the second scheduler 608 (e.g. using theage-based scheduling method described above) or alternatively thescheduled task may be switched from the PIP waiting state to a ‘waitingfor active’ state which is distinct from the ‘ready’ state describedabove into which scheduled tasks are placed initially prior to beactivated for the first time by the second scheduler 608.

In order for the PIP running list 622 to go from being full to not beingfull, a scheduled task has to be removed from the PIP running list 622and there are a number of different circumstances when scheduled tasksmay be removed. FIG. 8 is a flow diagram that shows two differentcircumstances when scheduled tasks are removed from the PIP running list622 and the method shown in FIG. 8 may be implemented by the instructioncontroller 620 and/or second scheduler 608. As shown in FIG. 8, when ascheduled task is de-activated for a reason other than to enter the PIPwaiting state (block 802), it is determined whether the scheduled task(that has been de-activated) is in the PIP running list 622 (block 806)and if the scheduled task is included in the PIP running list (‘Yes’ inblock 806), it is removed from the PIP running list 622 (block 808).Similarly, if a scheduled task completes (block 804), it is determinedwhether the scheduled task (that has completed) is in the PIP runninglist 622 (block 806) and if the scheduled task is included in the PIPrunning list (‘Yes’ in block 806), it is removed from the PIP runninglist 622 (block 808).

A further circumstance when a scheduled task may be removed from the PIPrunning list 622 is shown in FIG. 7. If it is determined that a decodedinstruction does not target the PIP (‘No’ in block 706), a check may beperformed to determine whether the scheduled task to which theinstruction relates is in the PIP running list 622 (block 726). If thescheduled task is in the PIP running list 622 (‘Yes’ in block 726), thescheduled task is removed from the PIP running list 622 (block 728).Irrespective of whether the scheduled task is or is not in the PIPrunning list, the decoded instruction is sent to the target ALU pipeline(block 408) as described above.

In the description above, the determination of whether the PIP list isfull (in block 712) and the determination of whether the PIP list is notfull (in block 720) uses the same criteria, i.e. whether the number ofscheduled tasks included in the PIP running list 622 equals a maximumnumber or not. In various examples, however, a different, lower numbermay be used when determining whether the PIP list is not full (in block720). This different test (in block 720, compared to block 712) may beused to provide hysteresis (i.e. such that the PIP running list 622 doesnot show as being not full immediately that the number of scheduledtasks in the list falls). Such hysteresis may, for example, allow ascheduled task to briefly perform some other work without its place inthe PIP running list being immediately taken by another scheduled task(i.e. giving a window of time during which a scheduled task may be ableto execute an instruction of indeterminate duration, and if it completesfast enough it is allowed back into the PIP running list).

Using the methods and apparatus for optimizing the scheduling ofscheduled tasks within a GPU or other parallel processing unit asdescribed above with reference to FIGS. 6-8, a proper subset of thescheduled tasks (i.e. those scheduled tasks in the PIP running list 622)are prioritized. This avoids large numbers of scheduled tasks makingslow, synchronized progress such that they complete simultaneously andinstead results in some scheduled tasks executing faster than others.The methods described result in the PIP being as full as possible withinstructions from a limited set of scheduled tasks which improvesefficiency. Using the methods described, high latency memory requestsmay be distributed more regularly (e.g. the number of scheduled taskswhich progress at any point is reduced towards the minimum numberrequired to keep the ALUs busy and the closer the number of scheduledtasks progressing is kept to the minimum number, the more evenlydistributed the high latency request become). Additionally the powerconsumption is reduced through reducing the number of scheduled taskswhich are re-activated (i.e. go from a waiting or ready state to anactive stage) only to be de-activated virtually immediately.

Methods and apparatus for software controlled scheduling (e.g.activation, deactivation and/or reactivation) of scheduled tasks withina GPU or other parallel processing unit can be described with referenceto FIGS. 9-10.

FIG. 9 is a schematic diagram showing a processor 900 which may be a GPUor other parallel processing unit. It will be appreciated that FIG. 9only shows some elements of the processor and there may be many otherelements (e.g. caches, interfaces, etc.) within the processor that arenot shown in FIG. 9. The processor 900 comprises a scheduler 902, aninstruction decoder 303 (which is part of an instruction controller 920)and a processing block 904.

The scheduler 902 comprises a first scheduler 106 that schedules tasksto form scheduled tasks and a second scheduler 908 that schedules (e.gactivates, deactivates and/or reactivates) scheduled tasks. As describedabove, tasks are generally scheduled only once by the first scheduler106; however, once a task is scheduled to form a scheduled task, thecorresponding scheduled task(s) may be scheduled (e.g. activated,deactivated and/or reactivated) many times by the second scheduler 908.In particular, there may be many scheduled tasks and only a propersubset of these scheduled tasks may be active (i.e. running andexecuting in the processing block 904) at any time. Consequentlyscheduled tasks may be activated and de-activated (e.g. by being placedinto one or more ‘waiting states’ where they are not active) many timesbefore a task is completed. The second scheduler 908 comprises one ormore queues 310 which store scheduled tasks 312 (which have the requiredresources and their dependencies met). In various examples the secondscheduler 908 comprises a single queue 310 and this queue 310 alsostores state data 314 for each of the stored scheduled tasks 312 (suchthat the state data 314 for a scheduled tasks 312 may be described asbeing associated with a scheduled task 312) and the state data 314 for ascheduled task 312 identifies the current state that the scheduled task312 is in, e.g. whether it is in an active, ready or waiting state andwhere there are multiple waiting states, which particular waiting stateit is currently in. In other examples, the second scheduler 908comprises multiple queues, each queue corresponding to a different stateand storing scheduled tasks that are in the particular state (e.g. a‘ready queue’, an ‘active’ queue and one or more ‘waiting’ queues).

The processing block 904 comprises hardware logic for executing theinstructions within scheduled tasks that are activated for execution bythe scheduler 902. The processing block 904 therefore comprises aplurality of ALU pipelines 316 and the ALU pipelines may be grouped inany way. The processing block 904 may comprise different types of ALUpipelines 316, e.g. with each type of ALU pipeline being optimized for aparticular type of computation. In examples where the processor 900 is aGPU, the processing block 904 may comprise a plurality of shader cores,with each shader core comprising one or more ALU pipelines 316. Invarious examples, the processing block 904 may be a single-instructionmulti-data (SIMD) processor (which may in various examples it may bereferred to as a Unified Shading Cluster (USC)) or a single-instructionsingle-data (SISD) processor.

The instruction controller 920 comprises one or more instructiondecoders 303.

As shown in FIG. 9, the scheduler 902 receives tasks 110 and the firstscheduler 106 selectively schedules these tasks 110 for execution by theprocessing unit 904. The scheduled tasks 312 are stored in a queue 310by the second scheduler 908 and then scheduled (e.g. activated,deactivated and/or reactivated) by the second scheduler 908. When ascheduled task 312 is activated by the second scheduler 908, its statechanges from a not active state (e.g. a ready state) to an active stateand it is sent to the instruction controller 920.

The instruction controller 920 receives active (i.e. activated)scheduled tasks from the second scheduler 908 and decodes instructionsassociated with the active scheduled tasks using the instruction decoder303. The instruction controller 920 then sends the decoded instructionsto the targeted ALU pipeline 316 (i.e. to the ALU pipeline 316 which canexecute the instruction). Each instruction has an associated swap flag(which may be set by software) and this flag controls whether the activescheduled task to which the instruction relates is de-activated or not.Operation of the swap flags and the instruction controller 920 aredescribed in more detail below.

FIG. 10 is a flow diagram of an example method of scheduling scheduledtasks within a GPU or other parallel processing unit which enablessoftware controlled scheduling. As shown in FIG. 9, the second scheduler908 activates scheduled tasks and sends instructions from activatedscheduled tasks to be decoded by the instruction controller 920 and thenexecuted by ALU pipelines 316 within the processing block 904. As shownin FIG. 10, in response to receiving an instruction for decoding (block402), the instruction controller 920 decodes the instruction (block 404,in the instruction decoder 303) and then checks whether the swap flag inthe decoded instruction is set (block 1006).

If it is determined (by the instruction controller 920) that the swapflag is not set (‘No’ in block 1006), then the decoded instruction issent to the target ALU pipeline 316 (block 408). If, however, it isdetermined (by the instruction controller 920) that the swap flag in thedecoded instruction is set (‘Yes’ in block 1006), then the instructioncontroller 920 triggers the de-activation of the scheduled task (block1010) and the decoded instruction is discarded. The de-activation isperformed by the second scheduler 908 (e.g. in response to a messagereceived from the instruction controller 920) and comprises switchingthe scheduled task from an active state back into the ready state (e.g.as recorded in the state data 314 associated with the scheduled task 312or by moving the scheduled task from an active queue into a readyqueue).

Once a scheduled task is de-activated and is in the ready state, it canbe activated again by the second scheduler 908 (e.g. using the age-basedscheduling method described above). On being de-activated, the scheduledtask goes to the back of the age-based queue 310 (i.e. the de-activatedscheduled task becomes the newest scheduled task in the queue 310).

The use of a swap flag as described with reference to FIGS. 9 and 10enables software (which specifies whether a swap flag is set or not foran instruction) to control the activation of scheduled tasks. This may,for example, be used to implement an alternative method of optimizingthe scheduling of scheduled tasks within a GPU or other parallelprocessing unit by prioritizing a subset (i.e. a proper subset) of thescheduled tasks to that shown in FIGS. 6-8 and described above. Thismay, for example, be implemented in software which sets swap flags bysetting the swap flag on the first instruction that does not target thePIP (after a set of PIP instructions). This has the effect thatfollowing the last of the instructions that targets the PIP, thescheduled task is de-activated and goes to the back of the queue 310. Invarious examples, through use of the swap flag, removal of a scheduledtask from the PIP running list 622 when a single instruction in asequence of instructions that otherwise targets the PIP does not targetthe PIP can be prevented. For example, in a sequence of instructions(for a scheduled task) which target the following ALU pipelines: PIP,PIP, PIP, PIP, non-PIP, PIP . . . , and where the non-PIP instruction(i.e. the instruction which does not target the PIP) is a simpleinstruction which does not take very long to execute on an ALU pipelinewhich is not the PIP, the swap flag may not be set for this instruction.This has the effect that the scheduled task is not de-activated and canimmediately revert to executing instructions on the PIP.

Use of the swap flag, in addition to any other scheduling method,provides additional flexibility and allows some software control overthe scheduling of instructions. It may also reduce the area of hardwarethat is required (e.g. by enabling hardware as described above withreference to FIGS. 6-8 to be omitted whilst still providing the samefunctionality). Furthermore, by reducing the number of scheduled tasksthat are reactivated (i.e. go from a waiting or ready state to an activestage) only to be de-activated virtually immediately, the methodsdescribed above with reference to FIG. 10 may also reduce the overallpower consumption of the apparatus.

Whilst the methods shown in FIGS. 2, 4 and 5, 7 and 8, and 10 aredescribed separately above, any two or more of the methods may be usedin combination (e.g. the methods described above with reference to FIGS.2, 4, 5, 7 and 8 may be implemented together and without the methoddescribed above with reference to FIG. 10) and as described above, themethod of FIG. 10 may be used to implement the method described withreference to FIGS. 7 and 8.

FIG. 11 shows a computer system in which the instruction controllerand/or second scheduler described herein described herein may beimplemented. The computer system comprises a CPU 1102, a GPU 1104, amemory 1106 and other devices 1114, such as a display 1116, speakers1118 and a camera 1120. A scheduler 1110, instruction controller/decoder1111 and processing block 1112 (corresponding to the schedulers,instruction controllers/decoders and processing blocks shown in any ofFIGS. 1, 3, 6 and 9) are shown implemented within the GPU 1104. In otherexamples, the scheduler 1110, instruction controller/decoder 1111 andprocessing block 1112 may be implemented within the CPU 1102. Thecomponents of the computer system can communicate with each other via acommunications bus 1122.

The processors of FIGS. 1, 3, 6 and 9 are shown as comprising a numberof functional blocks. This is schematic only and is not intended todefine a strict division between different logic elements of suchentities. Each functional block may be provided in any suitable manner.It is to be understood that intermediate values described herein asbeing formed by any element within a processor need not be physicallygenerated by the particular element at any point and may merelyrepresent logical values which conveniently describe the processingperformed by the processor between its input and output.

The instruction controller and/or scheduled task scheduler describedherein may be embodied in hardware on an integrated circuit. Theinstruction controller and/or scheduled task scheduler described hereinmay be configured to perform any of the methods described herein.Generally, any of the functions, methods, techniques or componentsdescribed above can be implemented in software, firmware, hardware(e.g., fixed logic circuitry), or any combination thereof. The terms“module,” “functionality,” “component”, “element”, “unit”, “block” and“logic” may be used herein to generally represent software, firmware,hardware, or any combination thereof. In the case of a softwareimplementation, the module, functionality, component, element, unit,block or logic represents program code that performs the specified taskswhen executed on a processor. The algorithms and methods describedherein could be performed by one or more processors executing code thatcauses the processor(s) to perform the algorithms/methods. Examples of acomputer-readable storage medium include a random-access memory (RAM),read-only memory (ROM), an optical disc, flash memory, hard disk memory,and other memory devices that may use magnetic, optical, and othertechniques to store instructions or other data and that can be accessedby a machine.

The terms computer program code and computer readable instructions asused herein refer to any kind of executable code for processors,including code expressed in a machine language, an interpreted languageor a scripting language. Executable code includes binary code, machinecode, bytecode, code defining an integrated circuit (such as a hardwaredescription language or netlist), and code expressed in a programminglanguage code such as C, Java or OpenCL. Executable code may be, forexample, any kind of software, firmware, script, module or librarywhich, when suitably executed, processed, interpreted, compiled,executed at a virtual machine or other software environment, cause aprocessor of the computer system at which the executable code issupported to perform the tasks specified by the code.

A processor, computer, or computer system may be any kind of device,machine or dedicated circuit, or collection or portion thereof, withprocessing capability such that it can execute instructions. A processormay be any kind of general purpose or dedicated processor, such as aCPU, GPU, System-on-chip, state machine, media processor, anapplication-specific integrated circuit (ASIC), a programmable logicarray, a field-programmable gate array (FPGA), or the like. A computeror computer system may comprise one or more processors.

It is also intended to encompass software which defines a configurationof hardware as described herein, such as HDL (hardware descriptionlanguage) software, as is used for designing integrated circuits, or forconfiguring programmable chips, to carry out desired functions. That is,there may be provided a computer readable storage medium having encodedthereon computer readable program code in the form of an integratedcircuit definition dataset that when processed (i.e. run) in anintegrated circuit manufacturing system configures the system tomanufacture an instruction controller and/or scheduled tasks schedulerconfigured to perform any of the methods described herein, or tomanufacture a processor comprising any apparatus described herein. Anintegrated circuit definition dataset may be, for example, an integratedcircuit description.

Therefore, there may be provided a method of manufacturing, at anintegrated circuit manufacturing system, an instruction controllerand/or scheduled task scheduler as described herein or a processorcomprising an instruction controller and/or scheduled task scheduler asdescribed herein. Furthermore, there may be provided an integratedcircuit definition dataset that, when processed in an integrated circuitmanufacturing system, causes the method of manufacturing an instructioncontroller and/or scheduled task scheduler as described herein or aprocessor comprising an instruction controller and/or scheduled taskscheduler as described herein to be performed.

An integrated circuit definition dataset may be in the form of computercode, for example as a netlist, code for configuring a programmablechip, as a hardware description language defining an integrated circuitat any level, including as register transfer level (RTL) code, ashigh-level circuit representations such as Verilog or VHDL, and aslow-level circuit representations such as OASIS(®) and GDSII. Higherlevel representations which logically define an integrated circuit (suchas RTL) may be processed at a computer system configured for generatinga manufacturing definition of an integrated circuit in the context of asoftware environment comprising definitions of circuit elements andrules for combining those elements in order to generate themanufacturing definition of an integrated circuit so defined by therepresentation. As is typically the case with software executing at acomputer system so as to define a machine, one or more intermediate usersteps (e.g. providing commands, variables etc.) may be required in orderfor a computer system configured for generating a manufacturingdefinition of an integrated circuit to execute code defining anintegrated circuit so as to generate the manufacturing definition ofthat integrated circuit.

An example of processing an integrated circuit definition dataset at anintegrated circuit manufacturing system so as to configure the system tomanufacture an instruction controller and/or scheduled task scheduler asdescribed herein or a processor comprising an instruction controllerand/or scheduled task scheduler as described herein will now bedescribed with respect to FIG. 12.

FIG. 12 shows an example of an integrated circuit (IC) manufacturingsystem 1202 which is configured to manufacture an instruction controllerand/or scheduled task scheduler as described herein or a processorcomprising an instruction controller and/or scheduled task scheduler asdescribed in any of the examples herein. In particular, the ICmanufacturing system 1202 comprises a layout processing system 1204 andan integrated circuit generation system 1206. The IC manufacturingsystem 1202 is configured to receive an IC definition dataset (e.g.defining an instruction controller and/or scheduled task scheduler asdescribed in any of the examples herein), process the IC definitiondataset, and generate an IC according to the IC definition dataset (e.g.which embodies an instruction controller and/or scheduled task scheduleras described in any of the examples herein). The processing of the ICdefinition dataset configures the IC manufacturing system 1202 tomanufacture an integrated circuit embodying an instruction controllerand/or scheduled task scheduler as described in any of the examplesherein.

The layout processing system 1204 is configured to receive and processthe IC definition dataset to determine a circuit layout. Methods ofdetermining a circuit layout from an IC definition dataset are known inthe art, and for example may involve synthesising RTL code to determinea gate level representation of a circuit to be generated, e.g. in termsof logical components (e.g. NAND, NOR, AND, OR, MUX and FLIP-FLOPcomponents). A circuit layout can be determined from the gate levelrepresentation of the circuit by determining positional information forthe logical components. This may be done automatically or with userinvolvement in order to optimise the circuit layout. When the layoutprocessing system 1204 has determined the circuit layout it may output acircuit layout definition to the IC generation system 1206. A circuitlayout definition may be, for example, a circuit layout description.

The IC generation system 1206 generates an IC according to the circuitlayout definition, as is known in the art. For example, the ICgeneration system 1006 may implement a semiconductor device fabricationprocess to generate the IC, which may involve a multiple-step sequenceof photo lithographic and chemical processing steps during whichelectronic circuits are gradually created on a wafer made ofsemiconducting material. The circuit layout definition may be in theform of a mask which can be used in a lithographic process forgenerating an IC according to the circuit definition. Alternatively, thecircuit layout definition provided to the IC generation system 1206 maybe in the form of computer-readable code which the IC generation system1006 can use to form a suitable mask for use in generating an IC.

The different processes performed by the IC manufacturing system 1202may be implemented all in one location, e.g. by one party.Alternatively, the IC manufacturing system 1202 may be a distributedsystem such that some of the processes may be performed at differentlocations, and may be performed by different parties. For example, someof the stages of: (i) synthesising RTL code representing the ICdefinition dataset to form a gate level representation of a circuit tobe generated, (ii) generating a circuit layout based on the gate levelrepresentation, (iii) forming a mask in accordance with the circuitlayout, and (iv) fabricating an integrated circuit using the mask, maybe performed in different locations and/or by different parties.

In other examples, processing of the integrated circuit definitiondataset at an integrated circuit manufacturing system may configure thesystem to manufacture an instruction controller and/or scheduled taskscheduler as described herein without the IC definition dataset beingprocessed so as to determine a circuit layout. For instance, anintegrated circuit definition dataset may define the configuration of areconfigurable processor, such as an FPGA, and the processing of thatdataset may configure an IC manufacturing system to generate areconfigurable processor having that defined configuration (e.g. byloading configuration data to the FPGA).

In some embodiments, an integrated circuit manufacturing definitiondataset, when processed in an integrated circuit manufacturing system,may cause an integrated circuit manufacturing system to generate adevice as described herein. For example, the configuration of anintegrated circuit manufacturing system in the manner described abovewith respect to FIG. 12 by an integrated circuit manufacturingdefinition dataset may cause a device as described herein to bemanufactured.

In some examples, an integrated circuit definition dataset could includesoftware which runs on hardware defined at the dataset or in combinationwith hardware defined at the dataset. In the example shown in FIG. 12,the IC generation system may further be configured by an integratedcircuit definition dataset to, on manufacturing an integrated circuit,load firmware onto that integrated circuit in accordance with programcode defined at the integrated circuit definition dataset or otherwiseprovide program code with the integrated circuit for use with theintegrated circuit.

Those skilled in the art will realize that storage devices utilized tostore program instructions can be distributed across a network. Forexample, a remote computer may store an example of the process describedas software. A local or terminal computer may access the remote computerand download a part or all of the software to run the program.Alternatively, the local computer may download pieces of the software asneeded, or execute some software instructions at the local terminal andsome at the remote computer (or computer network). Those skilled in theart will also realize that by utilizing conventional techniques known tothose skilled in the art that all, or a portion of the softwareinstructions may be carried out by a dedicated circuit, such as a DSP,programmable logic array, or the like.

The methods described herein may be performed by a computer configuredwith software in machine readable form stored on a tangible storagemedium e.g. in the form of a computer program comprising computerreadable program code for configuring a computer to perform theconstituent portions of described methods or in the form of a computerprogram comprising computer program code means adapted to perform allthe steps of any of the methods described herein when the program is runon a computer and where the computer program may be embodied on acomputer readable storage medium. Examples of tangible (ornon-transitory) storage media include disks, thumb drives, memory cardsetc. and do not include propagated signals. The software can be suitablefor execution on a parallel processor or a serial processor such thatthe method steps may be carried out in any suitable order, orsimultaneously.

The hardware components described herein may be generated by anon-transitory computer readable storage medium having encoded thereoncomputer readable program code.

Memories storing machine executable data for use in implementingdisclosed aspects can be non-transitory media. Non-transitory media canbe volatile or non-volatile. Examples of volatile non-transitory mediainclude semiconductor-based memory, such as SRAM or DRAM. Examples oftechnologies that can be used to implement non-volatile memory includeoptical and magnetic memory technologies, flash memory, phase changememory, resistive RAM.

A particular reference to “logic” refers to structure that performs afunction or functions. An example of logic includes circuitry that isarranged to perform those function(s). For example, such circuitry mayinclude transistors and/or other hardware elements available in amanufacturing process. Such transistors and/or other elements may beused to form circuitry or structures that implement and/or containmemory, such as registers, flip flops, or latches, logical operators,such as Boolean operations, mathematical operators, such as adders,multipliers, or shifters, and interconnect, by way of example. Suchelements may be provided as custom circuits or standard cell libraries,macros, or at other levels of abstraction. Such elements may beinterconnected in a specific arrangement. Logic may include circuitrythat is fixed function and circuitry can be programmed to perform afunction or functions; such programming may be provided from a firmwareor software update or control mechanism. Logic identified to perform onefunction may also include logic that implements a constituent functionor sub-process. In an example, hardware logic has circuitry thatimplements a fixed function operation, or operations, state machine orprocess.

Any range or device value given herein may be extended or alteredwithout losing the effect sought, as will be apparent to the skilledperson.

It will be understood that the benefits and advantages described abovemay relate to one embodiment or may relate to several embodiments. Theembodiments are not limited to those that solve any or all of the statedproblems or those that have any or all of the stated benefits andadvantages.

Any reference to ‘an’ item refers to one or more of those items. Theterm ‘comprising’ is used herein to mean including the method blocks orelements identified, but that such blocks or elements do not comprise anexclusive list and an apparatus may contain additional blocks orelements and a method may contain additional operations or elements.Furthermore, the blocks, elements and operations are themselves notimpliedly closed.

The steps of the methods described herein may be carried out in anysuitable order, or simultaneously where appropriate. The arrows betweenboxes in the figures show one example sequence of method steps but arenot intended to exclude other sequences or the performance of multiplesteps in parallel. Additionally, individual blocks may be deleted fromany of the methods without departing from the spirit and scope of thesubject matter described herein. Aspects of any of the examplesdescribed above may be combined with aspects of any of the otherexamples described to form further examples without losing the effectsought. Where elements of the figures are shown connected by arrows, itwill be appreciated that these arrows show just one example flow ofcommunications (including data and control messages) between elements.The flow between elements may be in either direction or in bothdirections.

The applicant hereby discloses in isolation each individual featuredescribed herein and any combination of two or more such features, tothe extent that such features or combinations are capable of beingcarried out based on the present specification as a whole in the lightof the common general knowledge of a person skilled in the art,irrespective of whether such features or combinations of features solveany problems disclosed herein. In view of the foregoing description itwill be evident to a person skilled in the art that variousmodifications may be made within the scope of the invention.

1. A method of scheduling instructions within a parallel processing unitcomprising: decoding, in an instruction controller, an instruction in ascheduled task in an active state; checking if an ALU targeted by thedecoded instruction is full by checking a value of an ALU work fullnesscounter stored in the instruction controller and associated with thetargeted ALU; in response to determining that the targeted ALU is notfull, sending the decoded instruction to the targeted ALU for executionand updating the ALU work fullness counter associated with the targetedALU; in response to determining that the targeted ALU is full,triggering a scheduler to de-activate the scheduled task by changing thescheduled task from the active state to a non-active state; monitoringthe ALU work fullness counters stored in the instruction controller; andin response to detecting, as a consequence of the monitoring, that anALU has changed from being full to not being full, triggering thescheduler to re-activate an oldest scheduled task waiting for the ALU byremoving the oldest scheduled task from the non-active state.
 2. Themethod according to claim 1, further comprising: checking if a flag inthe decoded instruction is set prior to checking if the ALU targeted bythe decoded instruction is full; and in response to determining that theflag in the decoded instruction is set, omitting the checking if the ALUtargeted by the decoded instruction is full and sending the decodedinstruction to the targeted ALU for execution without updating the ALUwork fullness counter associated with the targeted ALU.
 3. The methodaccording to claim 1, wherein checking if an ALU targeted by a decodedinstruction from a scheduled task in an active state is full by checkinga value of an ALU work fullness counter stored in the instructioncontroller and associated with the targeted ALU comprises: checking, inthe instruction controller, if an ALU targeted by a decoded instructionfrom a scheduled task in an active state is full by checking a value ofboth an ALU work fullness counter and a de-activated scheduled taskscounter stored in the instruction controller and associated with thetargeted ALU; and wherein the method further comprises: in response todetermining that the targeted ALU is full, updating the de-activatedscheduled tasks counter associated with the targeted ALU; and aftertriggering the scheduler to re-activate an oldest scheduled task waitingfor the ALU by removing the oldest scheduled task from the waitingstate, updating the de-activated scheduled tasks counter associated withthe ALU.
 4. The method according to claim 1, further comprising: inresponse to detecting that an instruction has been removed from a queuein a particular ALU, updating the value of the ALU work fullness counterstored in the instruction controller and associated with the particularALU.
 5. An instruction controller comprising: an input for receiving aninstruction in a scheduled task in an active state from a scheduled taskscheduler; an instruction decoder arranged to decode the receivedinstruction; a memory element arranged to store a plurality of ALU workfullness counters; and hardware logic arranged to check if an ALUtargeted by the decoded instruction is full by checking a value of theALU work fullness counter associated with the targeted ALU, and inresponse to determining that the targeted ALU is not full, to send thedecoded instruction to the targeted ALU for execution and update the ALUwork fullness counter associated with the targeted ALU; hardware logicarranged, in response to determining that the targeted ALU is full, totrigger the scheduler to de-activate the scheduled task by changing thescheduled task from the active state to a non-active state; and hardwarelogic arranged, to monitor the ALU work fullness counters and inresponse to detecting, as a consequence of the monitoring, that an ALUhas changed from being full to not being full, to trigger the schedulerto re-activate an oldest scheduled task waiting for the ALU by removingthe oldest scheduled task from the non-active state.
 6. The instructioncontroller according to claim 5, further comprising: hardware logicarranged to check if a flag in the decoded instruction is set prior tochecking if the ALU targeted by the decoded instruction is full; and inresponse to determining that the flag in the decoded instruction is set,to omit the checking if the ALU targeted by the decoded instruction isfull and to send the decoded instruction to the targeted ALU forexecution without updating the ALU work fullness counter associated withthe targeted ALU.
 7. The instruction controller according to claim 5,further comprising: one or more de-activated scheduled tasks counters;and hardware logic arranged to check if an ALU targeted by a decodedinstruction from a scheduled task in an active state is full by checkingif an ALU targeted by a decoded instruction from a scheduled task in anactive state is full by checking a value of both an ALU work fullnesscounter and a de-activated scheduled tasks counter associated with thetargeted ALU; and hardware logic arranged in response to determiningthat the targeted ALU is full, to update the de-activated scheduledtasks counter associated with the targeted ALU; and after triggering thescheduler to re-activate an oldest scheduled task waiting for the ALU byremoving the oldest scheduled task from the waiting state, to update thede-activated scheduled tasks counter associated with the ALU.
 8. Theinstruction controller according to claim 5, further comprising:hardware logic arranged, in response to detecting that an instructionhas been removed from a queue in a particular ALU, to update the valueof the ALU work fullness counter associated with the particular ALU. 9.A non-transitory computer readable storage medium having stored thereona computer readable description of an integrated circuit that, whenprocessed in an integrated circuit manufacturing system, causes theintegrated circuit manufacturing system to manufacture an instructioncontroller comprising: an input for receiving an instruction in ascheduled task in an active state from a scheduled task scheduler; aninstruction decoder arranged to decode the received instruction; amemory element arranged to store a plurality of ALU work fullnesscounters; and hardware logic arranged to check if an ALU targeted by thedecoded instruction is full by checking a value of the ALU work fullnesscounter associated with the targeted ALU, and in response to determiningthat the targeted ALU is not full, to send the decoded instruction tothe targeted ALU for execution and update the ALU work fullness counterassociated with the targeted ALU; hardware logic arranged, in responseto determining that the targeted ALU is full, to trigger the schedulerto de-activate the scheduled task by changing the scheduled task fromthe active state to a non-active state; and hardware logic arranged, tomonitor the ALU work fullness counters and in response to detecting, asa consequence of the monitoring, that an ALU has changed from being fullto not being full, to trigger the scheduler to re-activate an oldestscheduled task waiting for the ALU by removing the oldest scheduled taskfrom the non-active state.